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Using Reading Guides and On-line Quizzes to Improve Reading 
Compliance and Quiz Scores 


Abstract 

This study compared students’ daily in-class reading quiz scores in an introductory Child Development course 
across five conditions: control; reading guide only reading guide and on-line practice quiz; reading guide and 
on-line graded quiz, and reading guide and both types of on-line quizzes. At the beginning of class, students 
completed a 5-item quiz over the assigned readings. With the exception of the control section, all students had 
access to an instructor-designed reading guide for each of the 20 assigned readings. Results revealed that 
reading guides significantly increased student learning as demonstrated by increased scores on the in-class 
reading quizzes, with marginal additional gains when practice quizzes were also utilized. The addition of on¬ 
line graded quizzes resulted in lower scores on in-class quizzes. Results held even after multiple subsidiary 
analyses controlling for time spent studying. These findings suggest that reading guides maybe a valuable 
study aid for improving student learning. 
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Introduction 

The increased use of active learning strategies in classes 
requires students to be familiar with the content to be covered. 
Students cannot contribute in meaningful ways to discussions or 
participate in debates, role plays or other activities if they have 
not completed the assigned reading and comprehended the basic 
concepts under discussion (Koontz & Plank, 2011). Most 
professors believe that textbook reading increases students' 
understanding of course content "and that the more a student 
reads, the greater his/her facility with the content will be" 
(Vandsburger & Duncan-Daston, 2011, p. 6). However, empirical 
studies show that college students' reading compliance has 
declined substantially over the past 30 years from over 80% to 
less than 20% (Burchfield & Sappington, 2000). Data from the 
National Survey of Student Engagement [NSSE] indicates that 
over 80% of college seniors report attending class without 
reading or preparation (Nathan, 2005). Even in textbook-reliant 
introductory courses, students read less than one-third of the 
assigned pages (Gurung & Martin, 2011). Students who have not 
completed assigned readings on time are unprepared for class 
activities based on that material, making it difficult for 
instructors to move beyond content-delivery lectures. Solving 
the problem of low reading compliance requires that professors 
find new ways to motivate students to complete assigned 
reading. 

One strategy for encouraging reading compliance is a 
graded reading quiz, which provides an external incentive for 
doing the readings (Ruscio, 2001). Students are increasingly 
taking a consumerist approach to higher education, suggesting a 
shift from intrinsic to extrinsic motivations (Labaree, 1997), and 
a corresponding focus on external incentives. Recent work on 
students' academic motivations has revealed that students' 
strongest motivations are primarily extrinsic (Maurer, Allen, 
Gatch, Shankar, & Sturges, 2013, 2012), so connecting reading 
compliance directly to grades via reading quizzes may be an 
effective strategy to increase reading compliance. 

Further, short quizzes at the start of class, when paired 
with prompt feedback, are an effective teaching strategy, 
particularly in introductory courses (Connor-Greene, 2000). They 
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reduce "massed practice" (practice that occurs without rest 
between practices in contrast to "distributed practice") and 
procrastination (Maki & Maki, 2000), essentially diminishing the 
frequency of the pattern suggested by Gurung and Martin (2011) 
of waiting until just before an exam to do the assigned readings. 
Such quizzes also provide feedback to students on the 
effectiveness of their studying (Rosenthal & McKnight, 1996), 
which is far more useful before the first major exam, when 
changes can still be made that may improve performance on the 
exam. Moreover, reading compliance significantly predicts exam 
scores and final grades (Sappington et al., 2002). However, 
students also report that reading the textbook is one of the most 
common ways to study for course examinations (Gurung, 2005). 
In short, research findings suggest that quizzes motivate 
students to complete assigned reading when it is due rather than 
waiting until just before exams. 

In addition to motivating students to read before class, 
professors must also help students understand what they are 
reading (Koontz & Plank, 2011). Although faculty generally 
assume students have the skills need to handle reading 
assignments by the time they get to college, Kaback (2012) 
reports that some think students "don't have the skills to read 
effectively, so they don't even try" (p.21). National data support 
this perception. According to the most recent National 
Assessment of Educational Progress (NAEP), twelfth-grade 
students scored lower in reading in 2013 than they did in 1992, 
with only 38 percent performing at or above "Proficient." 
Shanahan and Shanahan (2008) write that "the proportion of 
students on track for successful college work actually diminishes 
as students advance through U.S. schools from eighth through 
twelfth grade" (p. 42). In fact, 21 percent of first-year 
undergraduate students at public institutions report taking 
remedial courses (Sparks & Malkus, 2013). Culver and Morse 
(2012) point out that the results of this perceived lack of literacy 
skill by students is that "many instructors have developed a 'sink 
or swim' policy, assuming that students will either gain the 
necessary skills or drop out" (p. 15). Given the declining 
graduation rate (Bound, Lovenheim, & Turner, 2009), this data 
suggests that more students are not developing the necessary 
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literacy skills and are dropping out. It has thus become 
necessary for professors to assume greater responsibility for 
supporting the reading development of students. This is 
predicated on the notion that any given text has discipline- 
specific demands which may be unknown to novice learners. 
Kaback (2012) posits that, as disciplinary experts, professors 
have experience navigating and interpreting text in their 
disciplines and are, therefore, "obliged to think of themselves as 
master artisans apprenticing their students to the craft of 
reading" (p. 19). Spencer and Jordan (1999) speculate that the 
pedagogic shift from the traditional teacher-centered to a 
student-centered approach "requires a fundamental change in 
the role of the educator from that of a didactic teacher to that of 
a facilitator of learning" (p.1280). 

Adding to this problem is the passive approach many 
students have towards reading assigned text. Instead of actively 
reading assignments, they typically focus on memorizing and 
"looking over" (Simpson & Nist, 1990), rather than actively 
engaging with the material or attempting to understand it. First 
and second year students—the primary population in many 
introductory courses—are especially likely to take such passive 
approaches, as they are less likely to be independent and self- 
regulatory learners (Cukras, 2006). Thus, the problem is not just 
getting students to complete the assigned readings on time, but 
getting them to engage with the readings in a meaningful way 
and learn from them. 

One way to help novice learners get more out of the 
readings and increase their motivation to complete reading 
assignments is the use of reading guides. Reading guides, also 
known as text guides, are teacher-developed resources that 
serve as "tutorials in print" (Holsgrove, Lanphear, & Ledingham, 
1998). They are designed to help students understand material 
as they read informational text. "As the name implies, [reading] 
guides provide students with the purposes and directions for 
reading a particular section or unit of text while the students are 
engaged in reading" (Montelongo, 2008, p. 289). Carefully 
prepared reading guides that present material "in a logical and 
accessible way can be an important aid to learning" (Holsgrove 
et al., p. 103). In addition to being beneficial for students, 
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reading guides can also support professors in the planning 
process because they require instructors to determine the 
learning outcomes associated with the assigned text. Once the 
learning outcomes are identified, the professor uses them to 
create a structured series of questions to guide students through 
the text, helping them to determine meaning and achieve basic 
comprehension and vocabulary (Horning, 2007). Reading guides 
"model how to select, decide, and focus upon what textbook 
material is important to learn" (Helms & Helms, 2010, p. 109) 
and "serve as a basis of lively classroom discussion, small group 
work, and a source of peer pressure to make sure students 
actually DO the reading" (Horning, 2007). As such, they can be 
viewed as a form of active learning and, as numerous research 
studies have demonstrated, active learning strategies are 
superior to lectures in promoting the development of students' 
problem solving and thinking skills (Koontz & Plank, 2011). 

From the student perspective, reading guides can be 
helpful in learning lesson objectives and preparing for graded 
assessments (Helms & Helms, 2010), and students who 
complete them score higher on graded assessments (Meiss, 
1983). These findings suggest that the combination of reading 
guides and daily reading quizzes may have a significant impact 
on both student reading compliance and student learning, but no 
prior study has simultaneously investigated the use of both 
methods. This study proposes to be the first. 

Additionally, this study proposes to incorporate on-line 
quizzes as an additional element to improve student reading 
compliance and student learning. On-line quizzes have been 
used in the prior literature investigating student learning 
(Brothen & Wambach, 2004; Daniel & Broida, 2004; Marchant, 
2002; Maurer, 2006) with mixed results, and on-line quiz scores 
have been reported to significantly predict scores on subsequent 
assessments over the same material, such as exams (Anthis & 
Adams, 2002). However, no prior investigation has used the 
combination of both on-line and in-class daily reading quizzes 
over the same material. This is a critical distinction because the 
degree of overlap between assessments is an important factor in 
determining the effectiveness of the quizzes (Burns & Vinchur, 
1992) and their utility in addressing other concerns, such as 
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illustrating the types of questions that will appear on future 
assessments and providing students constructive feedback on 
their level of mastery of the material (Thorne, 2000). According 
to the results of these studies, students who were given on-line 
quizzes before their daily in-class quizzes should perform better 
on the in-class quizzes, especially if the types of questions on 
the two assessments were similar in format. This study proposes 
to test that prediction. 

Specifically, this study will compare students' average daily 
reading quiz scores in an introductory child development course 
across five conditions: control, reading guide only, reading guide 
and on-line practice quiz, reading guide and on-line graded quiz, 
and reading guide and both on-line quizzes. When possible, 
controls for students' self-reported reading compliance, self- 
reported reading guide completion, and self-reported study 
hours, will be included. It is hypothesized: 

HI: Students in the four sections that receive 
reading guides will earn higher average daily reading 
quiz scores than students in the control section who 
do not receive reading guides. 

H2a: Students in the reading guide and on-line 
practice quiz section will earn higher average daily 
reading quiz scores than students in the control 
section or the reading guide only section. 

H2b: Students in the reading guide and on-line 
graded quiz section will earn higher average daily 
reading quiz scores than students in the control 
section, the reading guide only section, or the 
reading guide and on-line practice quiz section. 

H2c: Students in the reading guide and both types of 
on-line quizzes section will earn higher average daily 
reading quiz scores than students in the other four 
sections. 

Method 

Participants 

Participants were students in five sections of an 
introductory child development course taught by the first author. 
One section was taught each semester over a three year period. 
All five sections of the course met on Tuesdays and Thursdays 
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for 75 minutes each day over a 15 week semester. Students who 
withdrew from the course were excluded from data analyses, 
leaving a final sample of 290 students: control (n = 64), reading 
guide only (n = 79), reading guide and on-line practice quiz (n = 
78), reading guide and on-line graded quiz (n = 39), and reading 
guide and both on-line quizzes (n = 40). IRB restrictions 
prevented the collection of demographic data about the 
participants, but the modal participant was a traditional-aged 
white female who was taking the course to satisfy a requirement 
for a major or minor in Child & Family Development. 

Materials 

The primary dependent variable in this investigation was 
students' average daily reading quiz scores; missed quizzes were 
counted as zeros, consistent with course policy. Twenty quizzes 
of five items each were developed by the course instructor for 
each of 20 content days in the course. All quiz items covered 
material from the assigned readings for that day, approximately 
one half of a textbook chapter. All quiz items assessed learning 
at the "remembering" level of Anderson et al.'s (2001) 
Taxonomy. Over 90% of quiz questions were multiple choice, 
with the remainder being fill in the blank numeric questions. 
Additionally, the instructor created 20 on-line practice quizzes 
and 20 on-line graded quizzes of five questions each over the 
same assigned readings. Due to restrictions with the learning 
management system [LMS] software, all on-line questions were 
multiple choice. 

The instructor also created detailed reading guides for 
each of the assigned readings (20 total). All questions on the 
reading guide were organized in the order that students would 
encounter the answers in the text and contained a specific page 
number or numbers where the answers to that question could be 
located. Each reading guide required approximately two hours to 
develop and approximately two hours for a student to complete. 

The instructor also created a daily self-report survey for 
students to complete immediately before each day's daily 
reading quiz. This survey contained three questions: a) How 
much of today's assigned reading did you complete? b) How 
much of today's reading guide did you complete? c) How many 
hours did you spend studying for this day's class? The first two 
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questions used a five-point scale of: none, 25%, 50%, 75%, and 
All. The third question used a five-point scale of: <0.5, 0.5-1.5, 

1.5-3.0, 3.0-4.5, and >4.5. Average scores were computed for 
each of the three items across the semester. Missing data was 
counted as the lowest response option because anecdotal 
evidence suggested that students who missed class typically had 
not done the readings. 

Finally, the instructor created a seven-item end-of-course 
self-report survey. The first two items were identical to the first 
two items on the daily survey, except that they asked students 
to reflect on their reading compliance and reading guide 
completion for the entire semester. The third item asked 
students to report their average weekly time spent studying for 
the class over the semester, with response options double what 
appeared on the daily survey (e.g., <1 instead of <0.5). The 
remaining four items used a five-point Likert-type scale from 
Strongly Disagree to Strongly Agree and presented students with 
the statements: a) The reading guides were helpful in 
determining what to get out of the readings, b) The reading 
guides were helpful in preparing me for the class activities, c) 

The reading guides were helpful in preparing for the daily 
quizzes, d) The reading guides were helpful in preparing for the 
exams. 

Procedure 

Students in all five sections of the course received a 
closed-book, closed-note daily quiz over the assigned readings at 
the start of each of twenty content days in the course. Except for 
the control section, the daily quiz was immediately preceded by 
the daily self-report survey. The reading guide only section 
received only the first two questions on the daily self-report 
survey, whereas the other three sections to receive reading 
guides also received the question on hours studied. 

With the exception of the control section where reading 
guides were not used, the instructor explained how to complete 
and use the reading guides on the second day of class. The first 
assigned readings were due on the third day of class which was 
also when the daily quizzes began. All 20 reading guides were 
available from the first day of class in the on-line learning 
management system (LMS). 
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For the reading guide and on-line practice quiz and reading 
guide and both on-line quizzes sections, daily on-line practice 
quizzes opened immediately after each class period and 
remained open until 30 minutes prior to the next class period. 
Students could attempt each practice quiz only once and only 
during the time window when the quiz was open. After 
submitting their answers for the entire quiz, students would 
receive feedback about how many questions they correctly 
answered, which answer they selected for each question, and 
the correct answer for each question. For the reading guide and 
on-line graded quiz and reading guide and both on-line quizzes 
sections, daily on-line graded quizzes were administered in the 
same way. Both the practice and graded on-line quizzes were 
open book and open note. 

On the last day of the course, students completed the end- 
of-course self-report survey (except for the control section).All 
in-class quizzes, daily self-report surveys, and end-of-course 
surveys were administered via "clickers" (i.e., classroom 
response systems). For the in-class quizzes, the instructor 
displayed both the correct answer to each question and the 
distribution of responses from the class immediately after each 
question. For both of the surveys, the instructor did not display 
the distribution of class responses. 

Results 

Plan of Analysis 

Because this project used an incomplete experimental 
design, multiple methods of analysis were planned to explore 
potential differences between experimental groups and control 
for potential confounds. The first analysis will be a simple 
Analysis of Variance [ANOVA] using average daily reading quiz 
score as the dependent variable and condition as the 
independent variable. The second analysis will be an Analysis of 
Covariance [ANCOVA] adding as covariates three items from the 
end-of-course self-report survey: reading compliance, reading 
guide completion, and hours studied. Because the control 
condition did not receive the end-of-course survey, that group 
will be excluded from this analysis. The third analysis will be an 
ANCOVA using the daily self-report survey questions on reading 
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compliance and reading guide completion as covariates. Because 
the control condition did not receive the daily surveys, that 
group will be excluded from this analysis. The fourth analysis will 
be an ANCOVA using the daily self-report survey questions on 
reading compliance, reading guide completion, and study hours 
as covariates. Because the control condition and reading guide 
only condition did not receive the daily survey question on study 
hours, those groups will be excluded from this analysis. The final 
analysis will be a Multivariate Analysis of Variance [MANOVA] 
using condition as the independent variable (excluding the 
control condition) and the seven items on the end-of-course 
survey as dependent variables. 

Differences in Quiz Scores by Condition 

ANOVA. An ANOVA with condition as the independent 
variable and average daily quiz score as the dependent variable 
revealed a significant effect for condition, F (4, 285) = 18.19, p 
< .001, partial p 2 = .20. This effect size meets the criteria for 
"large" as established by Cohen (1988). Post hoc comparisons 
using least significant differences [LSD] revealed that most 
sections were significantly different from one another. The 
general pattern of results was that the reading guide only 
section and the reading guide and on-line practice quiz section 
had higher quiz scores than the control section, but the two 
sections that used on-line graded quizzes were not significantly 
different from the control section. Effect sizes for the contrasts, 
calculated using Cohen's d, were as follows: control < reading 
guide only (0.85), control < reading guide and on-line practice 
quiz (1.11), reading guide only > reading guide and on-line 
graded quiz (1.06), reading guide > reading guide and both on¬ 
line quizzes (0.61), reading guide and on-line practice quiz > 
reading guide and on-line graded quiz (1.33), reading guide and 
on-line practice quiz > reading guide and both on-line quizzes 
(0.88), and reading guide and on-line graded quiz < reading 
guide and both on-line quizzes (0.45). Most of these effect sizes 
meet the criteria for "large" effects as established by Cohen 
(1988). See Table 1. 
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Table 1 

Daily Quiz Means and Standard Errors by Section 


Section 

Control 
(N = 64) 


RG Only 
(N = 79) 

RG and 
Practice 
(N = 78) 

RG and 

Graded 
(N = 39) 

RG and Both 
(N = 40) 

M 

SE 

M SE 

M SE 

M SE 

M 

SE 

51.22 ab 

1.87 

63.80 c 1.69 

67.78 c 1.81 

48.00 a 2.40 

54.70 b 

2.37 


Note. RG = Reading Guide. Means with different subscripts significantly 
different at p < .05. 


ANCOVA with end of course survey. An ANCOVA with 
daily average quiz score as the dependent variable, condition as 
the independent variable, and three items from the end-of- 
course self-report survey as covariates (reading compliance, 
reading guide completion, and hours studied) was computed 
next. Again, a significant main effect for condition emerged, F 
(3, 178) = 23.63, p < .001, partial r\ 2 = .29, with a "large" effect 
size and the same general pattern of results by condition as the 
ANOVA analyses. Of the covariates, only self-reported reading 
guide completion significantly emerged as predictor, F(l, 178) = 
22.62, p < .001, partial p 2 = .11; r = .41, p < .001, consistent 
with prior research (Meiss, 1983). Neither self-reported reading 
compliance, r = .28, p < .001, nor self-reported study time, r = 
.07, ns, were significant predictors. 

To explore potential differences between daily and end-of- 
course self-reports, a series of three paired t-tests were 
computed. The first two tests, reading compliance and reading 
guide completion, compared students in all but the control 
condition, who did not receive those surveys. The third test, 
hours studied, compared students in the reading guide and on¬ 
line practice quiz, reading guide and on-line graded quiz, and 
reading guide and both on-line quizzes conditions only, as they 
were the only ones to receive that question on both surveys. All 
three tests were significant, with end-of-course self-report 
numbers higher than daily self-report averages. See Table 2. 
Because of these differences, ANCOVA analyses using daily self- 
reports were also necessary. 
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Table 2 

Daily and End-of-Course Self-Report Comparisons 



Report 









Daily 

End-of- 

course 







Variable 

M SE 

M SE 

t 

df 

P 

Cohen's 

d 

r 

P 

Reading 

compliance 

3.63 .06 

4.05 .06 

7.36 

190 

.000 

0.51 

.63 

.000 

Reading 

guide 

completion 

3.64 .08 

4.16 .07 

9.38 

188 

.000 

0.50 

.75 

.000 

Study hours 

2.78 .06 

3.17 .08 

6.16 

125 

.000 

0.49 

.65 

.000 


ANCOVA with daily surveys without study hours. An 

ANCOVA using the daily self-report survey questions on reading 
compliance and reading guide completion as covariates was 
computed next. Because the control condition did not receive the 
daily surveys, that group was excluded from this analysis. A 
significant main effect for condition emerged, F (3, 229) = 

20.87, p < .001, partial p 2 = .22. Again, the effect size was 
"large" and again the pattern of results by condition was the 
same. Both covariates emerged as significant predictors: self- 
reported reading compliance, F( 1, 229) = 37.89, p < .001, 
partial p 2 = .14, r = .73, p < .001; and self-reported reading 
guide completion, F (1, 229) = 32.66, p < .001, partial p 2 = .13, 
r = .74, p < .001. 

ANCOVA with daily surveys with study hours. An 

ANCOVA using the daily self-report survey questions on reading 
compliance, reading guide completion, and study hours as 
covariates was computed next. Because the control condition 
and reading guide only condition did not receive the daily survey 
question on study hours, both groups were excluded from this 
analysis. A significant main effect for condition emerged, F (2, 
149) = 25.20, p < .001, partial p 2 = .25. Again, the effect size 
was "large" and again the pattern of results by condition was the 
same. Both self-reported reading compliance, F(l, 149) = 
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19.59, p = .001, partial p 2 = .12, and self-reported RG 
completion, F (1, 149) = 21.83, p < .001, partial p 2 = .13, 
emerged as significant covariates. However, self-reported study 
time, ns, r = .59, p < .001, did not. 

Differences in End-of-Course Survey by Condition 

The final analysis was a MANOVA using condition as the 
independent variable (excluding the control condition) and the 
seven items on the end-of-course survey as dependent 
variables. A MANOVA was chosen because of significant 
intercorrelations between the seven dependent variables and to 
reduce the risk of Type I error. A significant multivariate main 
effect for condition emerged, Wilks' Lamba = .73, F (21, 474.34) 
= 2.57, p < .001, partial p 2 = .10, reflecting a "medium" effect 
size (Cohen, 1988). Significant univariate models emerged for 
only two items: self-reported reading compliance, F (3, 171) = 
4.15, p = .007, partial p 2 = .07, and perception of reading 
guides as helpful in preparing for exams, F (3, 171) = 3.57, p = 
.015, partial p 2 = .06. Both effect sizes were "medium." 

Students in the reading guide only section reported higher levels 
of reading compliance than students in the other three sections. 
As the number of graded assessments increased, students 
reported greater utility of the reading guides in preparing them 
for course exams. No significant differences were observed for 
reading guide completion, study hours, helpfulness of reading 
guides during reading, helpfulness of reading guides in preparing 
for class activities, or helpfulness of reading guides in preparing 
for quizzes. See Table 3. 


Discussion 

This study compared students' average daily reading quiz 
scores across five conditions: control, reading guide only, 
reading guide and on-line practice quiz, reading guide and on¬ 
line graded quiz, and reading guide and both on-line quizzes, 
controlling for students' self-reported reading compliance, self- 
reported reading guide completion, and self-reported study 
hours. The first hypothesis, that students in the four sections 
receiving reading guides would earn higher average daily reading 
quiz scores than students in the control section who did not 
receive reading guides, was partially supported. Results of the 
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ANOVA analysis revealed that students in the reading guide only 
and reading guide and on-line practice quiz sections did indeed 
score significantly higher on their daily reading quizzes than 
students in the control section. The average effect size for these 
two contrasts was nearly a full standard deviation (Cohen's d = 
1.00), an extremely large effect for SoTL research. However, 
students in both of the sections with on-line graded quizzes did 
not score significantly better than students in the control section 
on their daily reading quizzes. 


Table 3 

End-of-Course Survey Means by Section 



Section 









RG Only 

RG and 
Practice 


RG and 
Graded 


RG and 
Both 


Item 

M 

SE 

M 

SE 

M 

SE 

M 

SE 

Reading 

compliance 

4.40 a 

.11 

3.93 b 

.10 

3.94 b 

.14 

4.00 b 

.13 

Reading guide 
completion 

4.23 

.13 

4.31 

.13 

4.07 

.16 

4.03 

.16 

Hours studied 

3.04 

.11 

3.24 

.11 

3.36 

.14 

3.00 

.14 

Helpful—Readings 

4.38 

.16 

4.33 

.16 

4.27 

.20 

4.30 

.20 

Helpful—Activities 

4.23 

.13 

4.47 

.13 

4.36 

.17 

4.27 

.17 

Helpful—Quizzes 

4.51 

.12 

4.53 

.12 

4.30 

.16 

4.59 

.15 

Helpful—Exams 

4.06 a 

.12 

4.27 ab 

.12 

4.46 bc 

.16 

4.68 c 

.16 


Note. RG = Reading Guide. Means with different subscripts significantly 
different at p < .05. 


Hypotheses 2a, 2b, and 2c, which predicted improvement 
in students' daily quiz scores with the addition of on-line quizzes, 
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were not supported. Students in the reading guide and on-line 
practice quiz section did earn higher average daily reading quiz 
scores than students in the control section, but there was no 
significant difference between the scores for the reading guide 
only and reading guide and on-line practice quiz sections. 
Students in the reading guide and on-line graded quiz section 
actually earned lower scores than students in either the reading 
guide only or reading guide and on-line practice quiz sections, 
and equivalent scores to the students in the control section. 
Students in the reading guide and both on-line quizzes section 
did earn higher scores than students in the reading guide and 
graded quiz section, but lower scores than students in the 
reading guide only or reading guide and on-line practice quiz 
sections, and equivalent scores to the students in the control 
section. The pattern of results for all hypotheses held even after 
multiple subsidiary analyses (ANCOVAs) controlling for both daily 
and end-of-course self-reported reading compliance, reading 
guide completion, and study hours. 

Taken together, the results obtained here provided 
evidence that student performance on daily reading quizzes 
improved in a curvilinear fashion across sections, improving with 
the addition of reading guides (with or without on-line practice 
quizzes), but falling again with the addition of further graded 
assessments. The initial addition of the reading guides resulted 
in roughly a 13% raw increase in average student reading quiz 
scores (and a 25% proportionate increase). The addition of 
practice on-line quizzes resulted in a further roughly 5% raw 
increase beyond impact of reading guides (for combined 
proportionate increase of greater than 33%). Graded on-line 
quizzes, whether alone or in conjunction with practice on-line 
quizzes, appeared to fully offset the effect of reading guides and 
return quiz averages to that of control section. It is possible that 
because additional graded assessments decreased the relative 
contribution of daily reading quizzes to the final course grade, 
the addition of those on-line quizzes may have unintentionally 
disincentivized preparing for the daily reading quizzes. 

It is also possible that the addition of any on-line quizzes 
may have given students a false sense of confidence in their 
ability to score well on the daily reading quizzes, as student 
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reading compliance was identical in all three sections with on¬ 
line quizzes, but significantly lower than the reading guide only 
section. The on-line practice quizzes were designed to give 
students a way to test their knowledge and receive formative 
feedback (Rosenthal & Mcknight, 1996) without undermining the 
importance of the in-class quizzes for their grades. However, 
although students in the reading guide and on-line practice quiz 
section did score significantly higher than students in either of 
the on-line graded quiz sections, their reading compliance was 
identical. Because there was no difference between the sections 
in their self-reported study time, this suggests that the time 
students spent taking the on-line quizzes came from a fixed pre- 
established pool of time dedicated to studying the material, and 
that every minute spent taking the quiz(zes) was a minute not 
spent completing the readings, reading guides, or learning the 
material. Thus, if a zero sum approach to student study time 
was used, the feedback provided by the on-line quizzes would 
have had to have been superior to the benefits of spending that 
same time studying the material in other ways in order to have 
shown a difference here. The fact that it did not may suggest 
that additional quizzes have no significant value to student 
learning over other forms of student-selected study time. 

The results of the multivariate analyses suggested that 
students across the four sections that received reading guides 
did see significant value in them, regardless of the presence or 
absence of on-line quizzes. The average response to the last four 
questions on the end-of-course survey about the helpfulness of 
the reading guides in getting the most out of the readings, 
preparing for the daily activities, preparing for the daily quizzes, 
and preparing for the exams, was agree. It is highly unlikely that 
this was a mere halo effect for two reasons. First, the first three 
questions on the survey about reading compliance, reading guide 
completion, and hours studied were strongly correlated with 
daily reports, suggesting only minor inflation, if any. Second, the 
quiz average in all four sections was below a 70%, meaning that 
on average, students were failing the daily quizzes. Under these 
circumstances, a halo effect would be extremely unlikely. If 
anything, one would expect a negative response bias due to the 
cognitive dissonance associated with failing the quizzes, yet the 
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response to the reading guides was overwhelmingly positive (cf. 
Helms & Helms, 2010). These results suggest that reading 
guides could have real value in not only improving what students 
"get out of" assigned reading, but also in helping students to see 
the utility of study tools like reading guides particularly with 
respect to student learning. 

Limitations and Future Directions 

Like many SoTL investigations, this project has significant 
limitations. First, the organic design of the project—adding 
measures like study hours along the way as they became of 
interest—meant that there was partial missing data from two 
sections and that a full comparison could not be made. Ideally, 
an a priori design would have had all five sections collect daily 
and end-of-course self-reports on both reading compliance and 
study hours so that only two ANCOVA analyses would have been 
necessary to control for the potential impact of reading 
compliance and study hours on quiz grades across sections. 
Further, there was no way in this investigation to conclusively 
establish equivalency between sections in student abilities. It is 
possible that one (or more) sections had students of higher (or 
lower) ability than others. Although random assignment is 
seldom possible in classroom-based research, future 
investigations should at least collect additional data to control for 
potential pre-existing differences between students (e.g., GPA). 

Additionally, this investigation used one course in one 
discipline taught by a single instructor at a single university. 
Replication in other courses (especially textbook-reliant 
introductory courses) and other disciplines at other institutions is 
necessary to further explore the potential impact of reading 
guides and on-line quizzes on student reading compliance, 
performance, and learning. Some prior research has suggested 
that for first-year orientation courses that do not use reading 
guides or on-line quizzes, student reading compliance is not 
influenced by the presence, absence, or point value of reading 
quizzes (Maurer, 2011, 2010). Additional research has suggested 
that in specific contexts, students' academic behaviors may be 
immune to external incentives (Maurer et al., 2009). Given these 
findings, it is possible that reading guides and/or on-line quizzes 
may enable external incentives (such as reading quizzes) to 
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influence students' academic behaviors by increasing students' 
expectations for their own learning efficacy. For example, 
students with poor literacy skills may understand the importance 
of doing well on reading quizzes to their ultimate grade in the 
course, but may feel powerless to improve their quiz scores 
because their poor literacy skills prevent them from reading 
effectively. Reading guides that help such students develop their 
literacy skills may empower those students with a new sense of 
efficacy which allows them to successfully learn from the 
assigned readings and consequently perform better on reading 
quizzes. More qualitative investigations that probe how students 
use the reading guides and on-line quizzes and to what extent 
those uses really are active learning and literacy and efficacy 
promoting could be especially valuable (e.g., content analyses of 
completed reading guides, focus groups, interviews, etc.). This 
project was a collaboration between a faculty member and a 
faculty developer; such collaborations are ideally suited to these 
types of future investigations, as faculty developers can 
represent neutral parties to interview students about their study 
practices. 

Finally, it is important to note that the adoption of 
instructor-created reading guides comes at a significant cost 
which must be weighed against the benefit to student learning. 
Reading guides can be time-consuming to create (Helms & 
Helms, 2010; Shepherd, 2005). For the course in this 
investigation, the instructor had to create 20 reading guides. 

Each reading guide took approximately two hours to create, 
representing a full work week worth of effort to create them all. 
These reading guides would need to be completely recreated any 
time a new textbook (or new edition of a textbook) was adopted. 
Although reading guides appear to have significant benefits to 
student learning (Cherry, 2004; Shepherd, 2005), it is important 
to note that not all instructors may be able to invest the amount 
of "up front" time required to create them for a course and 
especially for multiple different courses. It is recommended that 
future research also explore how to "streamline" the process of 
creating reading guides to reduce the preparatory burden on 
instructors who wish to use them. One possibility is for 
instructors of the same course to work together to create both 
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the quizzes and the reading guides. Another is to work 
collaboratively with former students who have successfully 
passed the course to co-create reading guides. This latter 
approach is consistent with Felten's (2013) principles of good 
practice in SoTL in directly involving students as co-investigators 
in topics related to their own learning. 
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